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Abstract 



I n a comprehensive system of assessment/service delivery, there is a need for a set of tools that are 
useful for intervention planning within a decision-making framework. Such a system is under 
development by the Early Childhood Research Institute on Measuring Growth and Development. 
This report is one in a series describing the goals, results, and plans of the Institute. Discussed in this 
report are the research and development activities focused on a set of tools for planning interventions for 
individual children , birth to age 8. We have termed, this set of tools, Exploring Solutions Assessment, 
because of their unique role in a process of planning for interventions that accelerate the growth and 
development of children with disabilities. 



Introduction 



A basic tenet of special education is that educational services and interventions for children 
with special needs must be designed and evaluated based on objective assessment data 
(IDEA). The extent to which special education is guided by “intervention design” in actual 
practice, however, is debatable, and very likely related to limitations in existing assessment technologies 
and tools supporting decisions concerning interventions. In Technical Reports 1: Accountability Systems 
and 6: Theoretical Foundations of the ECRI-MD , we discussed in some detail the need for, and the 
conceptual framework underpinning the development of, a comprehensive system of intervention- 
focused assessment/service delivery. Unique to this system is its strong implications for monitoring of 
an individuals progress and planning and conducting interventions to improve progress. The system is 
designed for children aged birth to age 8, and it is an outgrowth of developments in performance-based 
assessment (e.g., Greenwood, 1996a,b) and curriculum-based measurement (e.g., Deno, 1985; Fuchs &C 
Fuchs, 1986; Shinn, 1989). The questions supported by this system of assessment include: Decid- 
ing whether or not to intervene, Deciding what behavior, knowledge, or skill will be the focus of 
intervention, Deciding where, when, with whom and how to intervene, Deciding if the interven- 
tion has been implemented, Deciding if the solution has been effective, and Deciding whether or 
not additional intervention is needed. 

Based on the general outcome measurement approach (GOM) (e.g., Deno, 1997; Fuchs &: Deno, 
1991), part of the system now under development focuses on the assessment of five superordinate 
outcomes and fifteen general growth outcomes (i.e., uses language - see Technical Report Report 2, 
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Appendix B). Just three advantages of the GOM approach compared to traditional approaches are: (a) 
that growth and development indicators (IGDIs) are assessed directly reflecting one’s progress within 
specific activities and/or curricula, (b) that growth is expressed as rate or slope; an intuitive, compel- 
ling framework that practitioners, parents, and students readily understand, and (c) that growth may 
be measured in the absence of a complete specification of all objectives and tasks in a teaching se- 
quence, as typically required in mastery-based or criterion-referenced assessment (Fuchs & Deno, 

1991 — see Technical Reports 4 and 6). While GOM provides excellent indicators of individual growth 
and development, it provides only limited information on what interventions are actually needed to 
increase rate of growth. 

Needed in addition to IGDIs is a varied set of tools to generate ideas for interventions capable of 
accelerating an individuals rate of progress in a truly comprehensive system. Termed “exploring solu- 
tions assessment” (ESA), these measures describe program features, curricular variables, and 
ecobehavioral interactions associated with growth and development, such that parents, practitioners, 
and others have information to monitor and rationally plan changes in the quality of intervention and 
other services” for individual children with disabilities (see Original Proposal, pg 9-1 1). Working 
together in an intervention decision-making framework (Barnett, McMann, & Carey, 1992; Deno, 
1989; Good & Kaminski, 1996), IGDIs and ESAs provide unique, complementary information 
needed for making decisions about progress versus lack of progress (i.e., Problem Identification, 
Problem Validation), and needed interventions (i.e., Exploring Solutions, Validating Solutions). Thus, 
the comprehensive system (IGDIs, ESAs, and their decision-making rules) is designed to optimize a 
child’s progress through a responsive, continually improving and adapting set of interventions, identi- 
fying those that work and adapting those that do not. An advance over current assessment practice 
with little direct relevance to intervention and progress decision making, this system is focused firmly 
on the progress of the individual child with direct implications for planning and conducting intervention. 

The purpose of this report is to: review the conceptual foundation of ESA, describe essential and 
desirable features of ESAs, describe the ECRI-MGD’s research and development process for develop- 
ing, selecting, and validating ESAs, and illustrate use of ESAs in the comprehensive system. 
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Purpose and Conceptual Foundation 



Objective 3 of the ECRI-MGD is to create assessments that describe program features, curricular 
variables, and ecobehavioral interactions associated with growth and development, such that parents, 
practitioners and others have information to monitor and rationally plan changes in the quality 
intervention and other services (Original Proposal, pp. 9-11). 

Conceptual Foundation 

The conceptual foundations guiding development of ESAs are based on several perspectives strongly 
supporting intervention and the active ingredients within interventions. Fundamentally, ESAs measure 
only “alterable variables” (Bloom, 1980) in the form of specially designed intervention, instruction, and 
care provided by teachers, parents, and caregivers within multiple settings (e.g., home, activity center, 
classroom). Variables that are not readily alterable (e.g., parents’ level of attained education), do not 
lend themselves to intervention in the same way that instructional/caregiving variables do to affect 
immediate change. 

ESA development is informed by an ecological perspective (e.g., Barnett, Carey, & Hall, 1993; 
Bronfenbrenner, 1979; Carta & Greenwood, 1985; Fox, 1990) because a child’s opportunity to receive 
intervention and to respond to it depends on one’s access to environmental settings, and the ability (or 
inability) of a setting to provide active supports for a child’s needs. ESA development is informed by an 
interaction perspective (e.g., Bijou & Baer, 1978) because the smallest observable units linking a child’s 
behavior and that of a parent or teacher within the act of caregiving or instruction is the reciprocal 
interaction. At the level of initiation and response, for example, a child’s behavior directly contacts the 
caregiver and the caregiving environment, each affecting one another (Bijou & Baer, 1978). 

“The interaction between the child and the environment is continuous, reciprocal, and 
interdependent. We cannot analyze a child without reference to an environment, nor is it possible 
to analyze an environment without reference to a child. The two form an inseparable unit 
consisting of an interrelated set of variables, or an interactional field (Bijou & Baer, 1978; pg. 29) 

ESA development is informed by knowledge of its relationship to the individual growth and develop- 
ment indicator (IGDI) that it is designed to impact. Central to ESA development is evidence that its 
use leads to accelerated development (i.e., treatment or consequential validity). Thus, in a comprehen- 
sive system, ESAs must be sensitive to potentially alterable variables that include: (a) specific environ- 
mental settings, (b) services, practices, routines, and activities available within these settings, (c) interac- 
tions children have with their caregivers, peers, materials, and physical objects in these settings, and (d) 
their use to plan and adapt interventions must be shown to covary with acceleration of individual 
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growth and development indicators. Lastly, to inform planning of what is needed most by a particular 
student , ESAs must be linked to what is known about effective intervention! instructional practice. We briefly 
discuss this essential knowledge base and its relationship to ESAs. 

Foundation in Effective Practice 

Results of a recent review of the intervention literature, those interventions and practices with strong 
evidence that they accelerate the learning of children with disabilities (Greenwood, Kamps et al., 1998) 
is summarized in Figure 1. As can be seen, factors in two classes of variables must be considered in the 
planning and use of interventions: (a) those that promote intervention effectiveness and (b) those that 
promote high quality implementation of these practices. It is these factors of intervention design and 
development , that ESAs are intended to direct practitioners in their efforts to plan individual interventions. 



Figure 1 . Conceptual Framework Describing Factors Known to Accelerate 




Factors essential to effective intervention/instructional practices 

The literature provides strong support for instructional method, curricular design, context manage- 
ment, behavior management, and use of instructionally relevant measures in the development of 
interventions that accelerate learning. 




Instructional method 

We know that children learn more when taught using specific methods. Example : Methods using peers and 
peer-mediated intervention/instruction have been reported to be more effective than teacher-mediated 
instruction (Mathes, Howard, Allen, & Fuchs, 1998; Wiedinger, 1998). Example : Methods based on 
incidental teaching are reported more effective than traditional approaches to accelerating use of lan- 
guage (e.g., Hart, 1985). 
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Curricular design 

We know that children learn more when taught with lessons and materials designed around * well connected " 
knowledge structures. Example: Explicit instructional strategies, including cognitive, direct instruction, 
and behavioral strategies, have led to increasingly powerful forms of instruction intervention (Becker, 
1977; Engelmann, 1997; Gersten, Carnine, & Woodward, 1987), and principles of instruction design 
(Carnine, Jones, & Dixon, 1994). Example : Explicit instructional strategies in peer tutoring, required 
tutors to present learning trials, check tutee response, and the provided differential forms of correction 
and reinforcement depending on the accuracy of response (Delquadri, Greenwood, Stretton, & Hall, 
1983). Example: Having students retell stories read, predict events based on current information, and 
check the accuracy of their predictions have been infused into teacher led instruction (Brown & 
Palincsar, 1989; Rosenshine & Meister, 1994) and classwide peer tutoring (Mathes, Fuchs, Fuchs, 
Henley, & Sandler, 1994; Simmons, Fuchs, Fuchs, Hodge, & Mathes, 1994). Example: Instruction on 
emerging literacy skills such as phonemic segmentation and letter naming (Good & Kaminsky, 1996; 
O’Connor, Notari-Syverson, & Vadasy, 1996) suggested that teaching these skills produced better early 
readers and reduced the probability of early reading problems. Mathes, Howard et al., (1998) demon- 
strated that these skills could be taught using peer assisted methods. 

Context management 

We know that home, school, and classroom contexts influence the time available to learn, the time devoted to 
specific subject matter, and childrens opportunities to respond. Example: Conventional conditions of 
teacher led, whole group instruction often resulted in unacceptably high rates of off task behavior and 
very low levels of active student engagement (Arreaga Mayer, Carta, & Tapia, 1994; Carta, Greenwood, 
& Robinson, 1987; Greenwood, 1996a; Ysseldyke, Graden, &Thurlow, 1982) in a variety of classroom 
settings, programs, and ages of students with special needs. These findings were replicated widely using 
ecobehavioral classroom observation systems (e.g., CISSAR, ESCAPE, and MSCISSAR Greenwood, 
Carta, Kamps, & Arreaga-Mayer, 1990; Greenwood & Delquadri, 1988). Example: Yet, under condi- 
tions of effective instructional intervention (e.g., peer tutoring or heterogeneous cooperative learning 
groups), powerful influences on the academic learning time and academic responding of students were 
demonstrated (Dugan, Kamps, & Leonard, 1995; Greenwood, 1991a,b; Kamps, Leonard, Potucek, & 
Garrison Harrell, 1995). 



Behavior management 

We know that engagement in academic responding, and low rates of inappropriate classroom behavior affect 
the acceleration of learning. Example: Rules and positive motivation have proven to be proactive compo- 
nents associated with accelerated learning outcomes (e.g., Forness et al., 1997). Example*. Functional 
analysis and positive behavioral support provided a new understanding of the function of students’ 
attention seeking and escape/avoidance from academic tasks (Dunlap & Kern, 1997; Kamps, Ellis et 
al., 1995; Lewis & Sugai, 1996). Example: School-wide management (Lewis, Colvin, & Sugai, in press) 
is a systematic way of coordinating staff expectations and conducting disciplinary procedures leading to 
fewer problems and more time spent engaged in academic learning. 
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Factors essential to sustained, quality implementation 

The literature provides strong support for the role of accessibility, usability, acceptability, training/ 
support, and caregiver understanding as factors promoting high quality use of specific interventions. 

Accessibility 

We know that ease of access to a needed intervention! instructional practice will influence its use. Example: 
Access to effective practice is moderated by the ease of acquiring it (Carnine, 1997). Example: Unfortu- 
nately, the best of instructional practices for preschool (Carta & Greenwood, 1997) and early elemen- 
tary grade levels is just not easily available to teachers (Carnine, 1997). 

Usability 

We know that the extent that an intervention is in procedural form, such that it can easily be set up and 
implemented in a step-by-step fashion using a manual, materials, and! or instructional scripts influences the 
use of a practice. Example: Carnine (1997) reported that because many research-validated practices are 
contained in research reports and journal articles, they are not in forms readily usable by practitioners. 
The usability of instructional interventions is often influenced by the nature and types of related 
research validated materials (Gerston, Vaughn, Deshler, & Schiller, 1997). 

Acceptability 

We know that caregiver acceptability (“comfort level”) with a curricular and instructional intervention is a 
potential threat to sustained use of an intervention. Example: King-Sears and Cummings (1996) assessed 
teachers’ level of comfort with multiple practices and directed efforts to increasing their knowledge and 
understanding of the practice. Example: Involving parents in the assessment and intervention design 
process supports their “comfort” with strategies selected to be implemented (Barnett et al., 1993) 

Training and support 

We know that traditional forms of teacher training too often fail to impact implementation of effective 
practices. Example : Professional development experiences provided in the home, the school and the 
classroom have shown far greater success sustaining use of effective instructional practices than have 
inservice workshops (Englert, Tarrant, & Rozendal, 1993; Gersten, Morvant, & Brengelman, 1995; 
Gersten et al., 1997; Slavin, Dolan, & Madden, 1996; Vaughn et al., in press). Example: It is known 
that parents and classroom teachers do not read research, and in fact teachers have a low regard for 
most educational research because of its limited relevance to classroom practice (e.g., Carta &C Green- 
wood, 1997; Viadaro, 1994). Coaching and collaborative consultation can bridge the gap between 
research and practice (e.g., Parents: Barnett, Carey & Hall, 1993; Teachers: Gersten &C Brengelman, 
1996). Example: Increasingly important are forms of collaboration between interventionists and 
caregivers that take place within the realities of the home, preschool, and public school classroom 
(Abbott, Walton, Tapia, & Greenwood in press, 1998; Barnett et al., 1993), and that provide problem 
solving based on efforts to improve instruction; and subsequently, child performance (Greenwood, 
Carta, Arreaga-Mayer, & Rager, 1991). 
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Teacher understanding 

We know that early childhood \ special , and general education teachers have a propensity for whole class forms 
of instruction and they rarely differentiate (adapt) their instruction by the needs of individual learner. 
Example: Only 25% of 1 10 general education teachers made any revision in their short-term instruc- 
tional plans for students with learning disabilities (Fuchs & Fuchs, 1998). Example: By selecting prac- 
tices with specific materials and concrete strategies and providing year long consultation and support at 
the school, 5 of 7 teachers sustained implementation of new practices over the school year (Vaughn, 
Hughes, Schumm, & Klingner, 1998). 

Summary 

Taken together in this brief review, it is reflection on these and other related factors of effective inter- 
vention/instruction that the ESA measures under development in this Institute are intended to support. 
Information directly relevant to the development of Individual Family Service Plans (IFSPs), prereferral 
interventions, and Individual Educational Programs (IEPs), as well as routine planning and instruc- 
tional decision making. This directing relationship can be seen in Figure 2. 

Here we have four classes of ESA measures: program features, activity/curriculum-based assessment, 
ecobehavioral assessment, and quality of implementation that point directly to specific intervention 
planning considerations previously discussed. Program features assessment provides the assessor a link to 
planning the environmental contexts of intervention/instruction. Activity! Curriculum-based assessment 
provides links to planning curricular design including ones strengths and weaknesses in the curriculum. 
Ecobehavioral assessment provides links to planning behavior management including specific categories 
of inappropriate and appropriate, engaged behaviors. Quality of implementation measures provide links 
to planning the training and support of the caregiver who will use the intervention. It also is a means of 
checking their understanding of the new procedures they are using. 



Figure 2. Linking ESAs to Intervention Planning Factors 



Exploring Solutions Assessment Intervention Planning Factors 



Program Features Assessment 

Activity/Curriculum-based 

Measurement 

Ecobehavioral Assessment 

Quality of Implementation 
Measures 





Instructional Method 
Curricular Design 
■^Context Management 
Behavior Management 
Instructional Relevant Measures 

Accessibility 

Usability 

Acceptability 

Training/Support 

Teacher/Caregiver Understanding 



[Footnote 1: ESAs are intended to direct users to factors of importance in the planning and implementation of interventions. It is not our intention to 
recommend specific interventions.] 
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Exploring Solutions Measures 

These four measures for exploring solutions represent two basic forms in the comprehensive system: (a) 
measurement that informs the goals and procedures of an intervention (program features , activity! 
curriculum- based assessment, and ecobehavioral assessment), and (b) measurement of the use of interven- 
tion, i.e., implementation of its component features, fidelity, and quantity (quality of implementation). 
These measures reveal what a child knows and can do, as well as the conditions in which the child is 
living and learning, including the services, care, and instruction they are or are not receiving (e.g., 

Kagen & Rosenkoetter, 1997). ESAs tell “what to do” (rather then “when to intervene” as do IGDIs) 
with respect to intervention planning (see Table 1). 

Compared to IGDIs which are narrow-band, selected indicators of general outcomes, ESAs are 
broad-band and comprehensive, identifying skills mastered versus those not mastered. ESAs are child 
focused as well as focused on the environment and caregiver in order to provide information related to 
a child and his/her conditions of learning. Compared to IGDIs, ESAs are measured intermittently in 
the absence of satisfactory progress, in order to yield information leading to prescriptions that inform 
the design of interventions to accelerate progress. 

Systems of measurement that have traditionally served these rules include: criterion-referenced 
assessment or mastery monitoring, curriculum-based assessment, curriculum-embedded assessment, 
play-based assessment, direct observation, and behavioral assessment (e.g., Bagnato, Neisworth, & 
Munson, 1989; Bricker, 1989; Notari & Bricker, 1990). These systems provide good models for our 
application. 

Some of the questions answered by ESA include: 

• How is the present environment organized (e.g., architecture, activities, persons)? 

• Are the desired activities and materials available in the setting and in use? 

• Is the caregiver located appropriately relative to the child? 

• How is the caregiver interacting with the child? 

• Do current features and caregiver behavior suggest a particular intervention? 

• Is a specific intervention taught to the caregiver in evidence compared to fidelity standards? 

• In the case of knowledge and skills (e.g., content being taught), does lack of mastery or error 
patterns suggest skills needing reteaching and additional practice? 

• In the case of skill hierarchies or sequences being instructed, what component skills are not 
established and need teaching or programming for generalization? 
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Table 1. Linkage between IGDI and ESA development by age and general outcome 



IGDI Development 


! ESA Development 


General 






{Program 


Activity-Based 1 


Ecobehavioral 


Outcome 


Age 


Indicator 


(Features 


Assess 


Assess 


1 . Expresses needs 


0-3 


Total Com. 


| X j 


1 x 


CIRCLE 


wants 


3-5 


Vocabulary 


! * 


* 


ESCAPE 




5-8 


Vocabulary j 


\ x 


* 


MS-CISSAR 


2. Cognitive/Literacy j 


1 0-3 i 


X ; 


! x j 


j X 


CIRCLE 


1 


| 3-5 


DIBELS 


X 


X 


ESCAPE 




5-8 | 


ORAL READ 


I X | 


! x 


MS-CISSAR 


3. Social 


0-3 | 


1 x | 


1 X I 


[ X 


! CIRCLE 




3-5 i 


X 


X 


X 


ESCAPE 


1 


r^n 


! x i 


- x 


X 


MS-CISSAR 


4. Adaptive j 


0-3 1 


| X I 


1 ^x 


I X 1 


| CIRCLE 




3-5 


X 


X I 


i X 


ESCAPE 




5-8 


X 


X | 


1 X 


MS-UISSAK 


5. Motor 


! 0-3 i 


X 1 


1 "K“ "1 


i X— 


CIRCLE 




3-5 


X 


1 X 


X 


ESCAPE 




5-8 


X 


' X 


X 


Mi5-UI£>i3AK 


Note: X = not yet developed; ’ 


r = developed 









Note: * = developed; x = development 
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Create a System of Tools for Intervention Planning 

Central to understanding a child’s learning environment is information about its key features. Program 
features assessment reflects the opportunity to learn certain skills as evidenced by a programs goals, 
philosophy, content, and/or curriculum targets. Program features assessment is targeted at the program 
level of analysis, not the child level. 

Central to understanding what to teach is information on the child’s strengths and weaknesses in 
the curriculum. Activity/curriculum-based assessment targets the child level of analysis, with the goal of 
identifying specific strengths and weaknesses within the activities, routines, and the academic curriculum 
of natural settings. 

Central to establishing behavioral goals for a child is information about behavioral deficits and 
problems in the natural setting. Ecobehavioral assessment targets the interaction between the child, 
caregivers/teachers, and the immediate surrounding environmental context. The goal of ecobehavioral 
assessment is to describe a child's engaged behaviors, enabling behaviors, inappropriate behaviors; 
caregiver/teacher behaviors; and situational learning contexts . 

Based on a review of the literature, this complement of ESAs appears both important and sufficient 
given the attributes we have assigned to them in the comprehensive system (e.g., alterable, ecological, 
interaction, valid, reliable, linked to sound intervention knowledge) and the intervention planning role 
we wish them to support. 

Create a System of Tools for Assessing Intervention Implementation 

Central to understanding the effects of an intervention is knowing that it has been implemented as 
intended; that is, the primary caregiver is using the intended procedures and the right amount of time 
(e.g., Carta & Greenwood, 1989). Central to conducting an intervention is training the relevant 
teachers and caregivers to use it. Assessing intervention implementation is a means of evaluating and 
informing this training effort (Carta & Greenwood, 1989; Wolery & Holcombe, 1993). 

Essential and desirable features 

Essential and desirable features of ESAs include the ability to: 

• Suggest recommendations or hypotheses for changing the caregiving/instructional 
plan of an individual child 

• Suggest potential effective instruction strategies and directions for which intervention 
procedures can be selected or developed and tested. 

• Define an intervention in procedural and objective terms 

• Provide information for changing or improving an intervention 

• Identify that a change in caregiving or instruction has occurred 

• Know that training of staff or a caregiver was successful 

• Determine whether an intervention remains in place 
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Criteria Adopted to Guide Development of ESAs 

• Direcdy link to child and family measures of growth and development 

• Focus on supports, services, and other environmental features of the child’s and family’s life 
that directly relate to growth and development in general outcomes 

• Reliability 

• Feasibility 

• Ease of understanding (proposal, pg 30). 
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Decision-making Model 



Individual Growth & 
Development Indicator (IGDI) 



IFSP/1EP Process Begins 
Family Agrees to Evalutaion 





Early Childhood Research Institute on 
Measuring Growth &c Development 



BEST COPY AVAILABLE 



13 



What is the role of ESA in the Model? 

As described in the summary of Report 6, and the flowchart describing the integration of IGDIs and 
ESAs in the decision making model, ESAs support the Exploring Solutions and Validating Solutions 
steps in the comprehensive system. ESAs are used to help identify explanatory hypotheses for the lack 
of adequate growth in an IGDI. ESAs provide the means of assessing and selecting among potential 
target skills, behaviors, and behavior classes, and their promoters in terms of program features, forms of 
instruction, routines, and interactions between child and caregiver. As potential intervention targets, 
ESAs typically assess classes of behavior known to be important to learning and instruction: (a) engage- 
ment, participation, social interactions, and compliance with the instructional program; (b) error 
patterns in the application of specific skills in the presence of standard evoking conditions - math sheet, 
response to a social greeting, etc.; and (c) status in a sequence or hierarchy of skills to be learned. 

What is the relationship of ESA to IGDI? 

As can be seen in flowchart, the relationship between ESAs and IGDIs is based on answers to a specific 
set of questions regarding lack of progress (IGDI) and the formation of solutions (ESA) and their 
success accelerating progress as measured subsequently in the IDGI. Used together for intervention 
decision-making (e.g., Deno, 1989), IGDIs and ESAs are a means of developing and testing data-based 
hypotheses about intervention targets, and the most likely effective teaching strategies. There relation- 
ship also supports determination of effective vs ineffective interventions leading to further adaptations 
as needed in a continuous progress monitoring system of care and special education. 

HOW DOES ONE CHOOSE AN ESA: THE EXPLORING SOLUTIONS INVENTORY 
The initial steps in the process of exploring solutions involves reflection on a set of questions that target 
the solution domain for which information is needed. Supporting this process is the Exploring Solutions 
Inventory (ESI). 

The ES Inventory under development will be a brief set of questions referenced to a specific child 
that reveals the depth of available information about and understanding of what the child knows and 
can do, and the child’s learning environment in three solution domains: strength/weaknesses in the 
curriculum, program features, and child/caregiver interactions. As indicated in the flowchart, results of 
the ESI might point to one or all of the solution domains, leading to administration of the relevant 
ESA(s) and analysis/interpretation of the information. For example, Does the child have an opportu- 
nity to perform the behavior? When, Where, and How often? Where in their learning conditions 
should opportunities occur? And, is the child supported and taught as needed? When answered, these 
questions lead to framing and trying out a potential solution (intervention) while evaluating its effects 
on the IGDI. 
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The Exploring Solutions Inventory: 

What do we know about this problem? 

• What is our history with this child? 

• Can we rule our a medical cause? 

• Do we know what the child knows, and can 
and can not do? 

• Do we know if this is a problem of excessive behavior? 

• Do we know the settings, activities, routines, and/or 
curricula of concert? 

• Do we know the extent of the caregiver’s support 
for these skills? 



Given ESA information, a solution is formulated in the context of the intervention planning factors 
previously discussed, and then implemented. The quality of implementation is assessed and feedback 
provided to the caregiver checking their understanding and focusing on aspects needing improvement 
until 90 % or higher levels of implementadon quality are achieved. If with implementation the solu- 
tion/intervention is observed to be successful (Is the solution working?), the evidence lies in the 
accelerated growth of an IGDI. 

In the face of continued progress, the Evaluating Solutions step (Is the intervention effective?) 
supports a decision related to the degree of success or failure to improve, and the extent that the initial 
problem has been solved. Should an ESA informed intervention fail to improve growth in an IGDI, 
revisiting ESA data can support the next step of redesigning additional changes in care/instruction 
based on new information from supplied by recycling through the process to answer new questions 
related to finding a solution. 
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Exploring Solutions Assessments 



Program Features 

What do we know about the active ingredients of this ESA that Affects 
caregiving, teaching and learning? 

A number of authors have noted the effect that programmatic variables have on the developmental 
outcomes of children (George, George & Groesnick, 1990; Sechrest & Figuerodo, 1993; Snyder & 
Sheehan, 1996). While there is not an extensive knowledge base illustrating the impact of specific 
program features on an individual child’s development, it is probable that services for infants, toddler, 
preschoolers and early elementary- aged children vary greatly in a number of programmatic characteristics. 

Infant/Toddlers 

For infants and toddlers, these program features may include access to toys and materials, quality and 
quantity of parent-caregiver interaction, or access to health care. 

Preschoolers 

For preschool-aged children, the quantity of materials or lower-student teacher ratio appear to be 
critical program features. 

Early Elementary 

For early-elementary- aged students these critical program features may include organization of class- 
room space, access to literacy-based materials, scheduled time for interaction with peers. 

Our assumption is that these program features are critical and important aspects of a setting and 
significantly influence the growth and development of children. Despite the acknowledgment that 
program features effect child behavior, there is litde, if any, information on the individual or combined 
effects of these features on the growth and development of individual children. 

Research plans and strategies for shaping this instrument from pre-existing, 

RELATED TOOLS INTO FINAL PRODUCTS. 

The work of ECRI-MGD researchers during Year 1 of the Institute has been focusing on identification 
of critical program features in pre-school classroom that impact language development. As a result of an 
extensive literature review, several key program features have been identified. First, it is clear that class 
size and teacher student ratio impact language development. Smith and Dickinson (1994) found that 
lower class sizes leads to more pretending, cooperative play, and talking during freeplay and generally 
better quality child-child interaction throughout the day. Similar results have been noted by other re- 
searchers (e.g., Dunn, Beach, &C Kontos, 1994; File &C Kontos, 1993; Phillips, McCartney, & Scarr, 1987). 
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Second, program features that promote and support early literacy also impact language develop- 
ment. For example, access to role-play centers with familiar reading materials has been shown to 
increase responding to questions and increase language use, comprehension, and appropriate participa- 
tion during storytime (Martinez & Roser, 1985; Morrow, 1988). Other literacy-related program 
features include high quality reading area with an abundance of a wide-range of books and other 
literacy materials that are changed frequently (Watson, et al., 1994); integration of literature and 
literacy materials during freeplay activities (Kleeck, 1990) and displays of childrens art, names, sched- 
ules, and labeling of classroom items (Taylor, et al., 1986). 

We have completed an extensive literature review of Program Features in preschool classrooms that 
impact language development. In addition, we have reviewed a number of observation-based instru- 
ments including the Early Childhood Environmental Ratting Scale (ECERS) (Harms & Clifford, 
1980) Based on the results of these reviews, we identified 6 different language-based program features 
(broadly described above): 

1. smaller class and group size with lower teacher-student ratio 

2. abundance of books and reading materials that are rotated frequently 

3. high quality reading corner with soft and carpeted areas 

4. integration of literature and literacy materials during freeplay activities 

5. adult model who is frequently writing, reading and using print in the classroom 

6. displays of childrens work, labeling of classroom items and schedule 

These features were then reviewed by members of our Institute team at the three sites as well as by 
two researchers in early language development. These reviewers validated the results of our literature 
review based on their own professional judgement and knowledge of the research. In addition, they 
suggested specific questions that might assist observers in further defining each feature. 

The information from the literature review and professional expertise is included in a classroom 
observation form we developed (see Appendix A). This form includes broad questions which an 
observer or teacher answers after spending a brief period of time observing in a preschool classroom). 
In addition, teachers complete a brief questionnaire. The observation/questionnaire will be pilot tested 
in three types of classrooms: language-based early childhood special education classrooms, NAEYC 
approved preschool classrooms, and Head Start classrooms to determine the extent to which the 
identified program features are present in these three settings. Future research will assess the relation- 
ship between the presence of specific Program Features and an individuals child’s rate of language 
growth and development. 

Finally, we will follow the same process (literature review and expert panel) to design Program 
Feature observation instruments across the age spectrum (infants, toddlers, pre-school and early 
elementary), for different setttings (home or school), and across developmental or academic domains. 
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Activity / Curriculum-Based Assessment 

What do we Know about the Active Ingredients of this ESA That Effects 
Caregiving, Teaching, and Learning? 

Within an interactional/ecological model of development, human development is viewed as the result 
of ongoing reciprocal interactions between an organism and the environment over time. Sameroff and 
Chandler (1975) describe relationships between individuals and their environments as transactional, 
that is, relationships are reciprocal and influence one another in a bidirectional fashion. Thus, not only 
is a child’s development influenced by his/her environment, but the behaviors and developmental 
competencies of a child influence his/her environment and the way in which caregivers respond to the 
child in turn. 

Sameroff and Fiese (1990) recently described three preventive early intervention strategies that 
incorporate transactional concepts. These strategies focus on (a) changing the child, with eventual 
changes occurring in the caregiver (remediation), (b) changing the behavior of the caregiver, with 
eventual changes occurring in the behavior of the child (reeducation), and (c) changing caregiver-child 
interactions to facilitate more optimal caregiving interactions (redefinition). All three intervention 
strategies should be utilized within an ecological intervention model. Activity-Based Assessment is 
designed to provide information related to the first of these preventive strategies, that is, those focused 
on the child. 

When a child is identified as needing intervention, one of the first intervention decisions to be 
made is, “What should be the goals and focus of intervention?” To answer that question, it is not 
enough to know, for example, that a 3-year old child is not developing at pace with his/her peers or is 
not yet talking in sentences. While speaking in sentences may be the long-term outcome, the focus of 
intervention may be quite different. Questions that need to be addressed before intervention goals and 
focus can be determined include, “How does the child communicate his wants/needs? Does the child 
use gestures? word approximations? inconsistent vocalizations?” It would be developmentally inappro- 
priate to focus on the production of sentences in intervention when a child does not yet have the 
necessary preskills of communicating in single words or short phrases. 

The ABA approach under development in ECRI-MGD assesses a child’s functional skills across 
developmental domains within contexts that are close to routine/familiar. The activities are designed to 
evoke behaviors and skills related to General Growth Outcomes and vary across age groups from 
infancy to early elementary age. For example, infant/ toddler activities may consist of free-play with 
simple toys or interactions with caregivers; preschool activities may include block play and art activi- 
ties; early elementary activities may include reading from a book in the school curriculum. As a result, 
the ABA activities tap those child-related behaviors and skills that are prerequisites or components of 
General Outcomes and therefore are “active ingredients” with respect to designing interventions. 

Skills related to the General Growth Outcome, The child uses language to convey and comprehend 
communicative and social intent, will be used to provide examples of child related “active ingredients” 
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that are important for planning intervention goals and procedures. Knowing what skills a child already 
has, and which skills he/she needs to develop is information that will help to guide the focus and 
procedures of intervention. 

htfant/Toddlers 

If an IGDI indicates that a child is not showing growth related to language development because he/she 
is not using words, ABA provides a context for assessing a broader range of communcative skills the 
child does have and the situations in which he/she uses the skills. Additional skills observed within 
different analog activities might include documentation of vocalizations and gestures that the child 
uses, as well as the communciative functions served by the skills. For example, across various routine 
play and caregiving activities, the child may use gestures to request objects and help and vocalizations to 
seek attention. In this case, the focus of intervention might be on increasing child’s use of vocalizations 
for a variety of functions. 

Preschool 

For a preschool child, additional skills assessed might include related communicative skills (e.g., use of 
vocalizations, single words, multiple word utterances) as well as related skills such as play behaviors and 
social skills. For example, childrens talk has found to be accelerated when a child’s play skills are at the 
symbolic, rather then the sensori-motor level of play. If ABA assessment indicates that a preschool 
child’s play skills are at a sensori-motor level, an intervention approach to increasing communication 
could be to increase symbolic play skills by using symbolic play activities as a context for communica- 
tive interventions. 



Early Elementary 

For children in elementary grades, additional skills and competencies assessed through ABA might 
include a broad range of more sophisticated communication skills, including child’s use of different 
word forms and grammatical structures in connected discourse. In addition, related skills such as early 
literacy skills and or social skills may be assessed in the context of picture book reading. 

“Developmental assessment is a process designed to deepen understanding of a child’s competencies 
and resources, and of the caregiving and learning environments most likely to help a child make fullest 
use of his or her developmental potential.” (Greenspan & Meisels, 1996). ABA provides a format for 
gathering information about the child’s skills and competencies within a developmental hierarchy or 
developmental/academic curriculum in order to design interventions that will promote the child’s 
growth and development. The following questions may be answered through the use of ABA: 

1 . What is the child’s performance on skills and competencies related to the General Outcome? 

2. What is the child’s performance on skills and competencies related to the IGDI? 

3. What are the child’s errors or patterns of errors on skills related to the IGDI? 

4. What adaptations and/or assistance does the child require to perform the skill? 

5. What is the child’s performance on related skills in other domains? 
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Research Plans and Strategies for Shaping this Instrument from 

Pre-existing Related tools into Final Products: Development of Curricular/Play- 

Based Assessment 

The research proposed in this application is focused on an assessment approach that has three critically 
important features: 1) the approach uses analog activities as the primary assessment format, 2) the 
information generated can be linked directly Individual Growth and Development Indicators (IGDIs) 
for young children, and 3) the information generated can provide useful information regarding child 
skills and competencies that may impact interventions for young children. Development of a curricu- 
lar/play based approach to assessment is currently underway. Procedures are being selected, developed, 
and piloted to reflect and merge current best practice in both curriculum-based assessment and play- 
based assessment for young children and will be developed to meet the following criteria: 

1. involvement of families will insure that parents are significant partners in the assessment process 
and that the home environment is taken into account when important intervention targets and 
strategies are being considered; 

2. multi domain assessment will ensure that intervention targets and strategies reflect the fact that 
developing skills are undifferentiated and intertwined holistically across developmental domains 
in meaningful contexts (Fewell & Glick, 1993). 

3. trans disciplinary approach will ensure that teams address collaboratively the functional needs of the 
children they work with rather than focus on domain specific skills. This approach necessarily 
involves increased interaction and efforts to reach consensus among team members in an effort to 
produce assessment data and integrated goals that are derived from combinations of domains. 

4. linkage to IGDIs during typical routine, instructional, and play activities will ensure that 
intervention targets and strategies to promote growth and development will be ecologically valid. 

3. structure and standardization of administration and scoring procedures will provide a replicable 
procedure leading to the collection of more efficient , precise, and valid data about a child’s skills 
and competencies for intervention planning. 

Development of a curricular/play based procedure will require developing and piloting procedures 
to establish assessment. For example: 

• In which activities can a variety of skills be embedded across competencies and age cohorts? 

• What materials and what degree of structure/format elicit the maximum responses from children 
across competencies and age cohorts? 

• What modifications in activities and materials are necessary to accommodate developmental 
differences across the age cohorts? 
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Evaluation of Activity / Curriculum-Based Assessment Measures 

Following development and refinement of the ABA procedures, a series of evaluation studies will be 
undertaken to investigate the reliability, validity, cost-effectiveness, and treatment validity of the proce- 
dures. Studies will be conducted across competencies as well as within and across age groups and 
subgroups. 

Sibjscts 

Subjects will be children between the ages of birth and 8 years of age with children evenly distributed 
across Institute age cohorts (i.e., infants, toddlers, preschoolers, early elementary age). Subjects will be 
recruited from agencies serving young children and from public and private elementary schools and will 
include all of the relevant subgroups as identified through the Consensus Building process. 

Measures 

Primary measures will include the Curricular/Play Based Assessment procedures, IGDIs, and ESAs 
(Programmatic Assessment and Observational) for each Outcome and age cohort. Additional concur- 
rent validity measures may be utilized for particular Outcomes and/or age cohorts as needed to address 
research questions. 

Data Collection 

An overview of the types of studies that will be conducted for evaluation of ABA is provided in Table 1. 



Table 1. Overview of Studies to Evaluate Activity-Based Assessment Measures 



Focus of 
Research 


Research Questions 


Studies 


Reliability 


1 . What is the stability of ABA measures 
across a two-week period of time? 

2. What is the congruence of scores on 
ABA between 2 independent raters? 


1. Administer measures 2 weeks 
apart 

2. Independent raters observe and 
score each assessment 


Construct Validity 


1. What is the relationship between 

performance on ABA and performance 
on IGDIs for all age cohorts? 


1. Concurrent administration 
of ABA and IGDIs 


Cost Effectiveness 


1 . What is the cost of collecting ABA data 
for each age cohort? 


1. Ongoing collection of data 
regarding time/cost etc. 


Treatment Validity 


1. Does use of ABA lead to development 
of effective interventions for children 
in each age cohort? 


1. Combination of single 
subject and group research 
comparing effects of ABA 
on outcomes for children 
in each age cohort 
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The reliability studies will focus both on stability of assessment procedures over time and congruence 
between two scorers of the assessment. Stability of the procedures will be examined across a 2-week 
time frame for all age cohorts. 

Construct validation studies will ensure that the ABA procedures are related to IGDIs across compe- 
tencies and across age cohorts. The Activity- Based Assessment will be administered to subjects across 
age cohorts concurrent with the administration of IGDIs for the respective cohorts. 

The studies of cost effectiveness will ensure that the ABA procedures are feasible to implement. 
Throughout the evaluation studies, a Cost Analysis Log will be maintained by project staff document- 
ing: (a) cost of assessment materials, (b) time required for administration of assessment, (c) time 
required for scoring assessment, (d) time required for training assessors. 

Because the ABA procedures are being developed to provide a linkage to intervention and lead to 
the development of effective interventions, treatment validity studies will be the major focus of this 
group of research. A brief description of the treatment validity studies is provided below. 

TreatmentValidity 

To evaluate the treatment validity of the ABA, a series of single subject and group research will be 
undertaken. Research activities will be conducted across all age groups. The purpose of these research 
activities will be to document the effects of use of the ABA procedures by practitioners on intervention 
outcomes for children. 

Examples and Illustrations 

What is the focus of intervention (ue., what behaviors are to be taught/ learned?) ABA provides rich infor- 
mation regarding the behaviors/skills to be taught to a child. As discussed earlier, when a general outcome 
measurement system is used for determining the effectiveness of interventions, the focus of intervention 
is not necessarily the same as the outcome. It is critical to have information regarding the child’s current 
level of skill and competence in the curriculum so that appropriate intervention can be designed. 



AEPS Skills/Competencies 

• Organized by Domains 

• Social Communication Domain (3 to 6)? 
- G4 Uses pronouns 

• 4.1 Uses subject pronouns 

• 4.2 Uses object pronouns 

• 4.3 Uses possessive pronouns 

• 4.4 Uses indefinite pronouns 

• 4.5 Uses demonstrative pronouns 
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For example, if a three-year-old child is not yet talking, the focus of intervention should be deter- 
mined by assessing the child’s current communicative skills and patterns. If the child uses gestures, the 
focus of intervention may be to increase the child’s use of vocalizations paired with gestures. If the 
child vocalizes, the focus of intervention maybe to shape vocalizations to word approximations. If the 
child uses single words, it may be appropriate for the focus of intervention to be to expand the child’s 
single word utterances to two- and three-word utterances. 

Assessing skills in related areas also may provide useful information for determining the appropriate 
focus of intervention. For example, an observation of a child’s social skills may indicate that the child 
has low levels of social interactions with peers. Intervention may focus on providing more opportuni- 
ties for social interaction in order to increase both social skills as well as opportunities to use commu- 
nication skills. 

What caregiving! treatment! instructional strategies and arrangements will be used for the intervention ? In 
addition to determining focus of interventions, the design of intervention strategies also can be sup- 
ported by the use of ABA. Because ABA is conducted during activities which are as close as possible to 
typical routine caregiving and instructional activities, numerous opportunities are available to try out 
different prompts/cues and modifications of stimuli and reinforcers to see which work best to elicit the 
behavior and to facilitate performance of the behavior. For example, during ABA different strategies 
may be utilized to determine which ones best elicit the production of vocalizations from a child. As a 
result, it may be observed that a child consistently vocalizes when he/she wants to obtain a toy or 
other item that is within site but out of reach. This strategy could incorporated into an intervention 
plan for increasing vocalizations of a nonverbal child. 
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Ecobehavioral Assessment of Child/Caregiver Interactions 
What do we know about the active ingredients of this ESA that Affects 

CAREGIVING, TEACHING, AND LEARNING? 

The behavior of an individual in interactions with caregivers and teachers in a learning context over 
time is considered one causal process influencing growth and development (Greenwood et ah, 1992). 
In concept, these processes are described as ecodevelopmental (Bradley et al., 1994) or as ecobehavioral 
(Morris & Midgely, 1990). Thus, a child’s behavior, and his care and instruction, when assessed by 
instruments based on these frameworks become observable, measurable events with potential for 
informing plans to accelerate growth and development (e.g., Wolery, 1996). Instruments used in this 
Institute to assess these processes are the observational taxonomies and instruments developed at the 
Juniper Gardens Childrens Project. These instruments are: 

• Code for Interactive Recording of Caregiving and Learning Environments 

(CIRCLE 1 & 2) - Infancy (Baggett et al., 1997) to Early Childhood (Atwater, Montagna, 
Creighton, Williams, & Hou, 1993), 

• Ecobehavioral System for Complex Assessments of Preschool Environments (ESCAPE) - Preschool 
(Carta, Greenwood, &, Atwater, 1985), and the 

• Code for Instructional Structure and Student Academic Response : Mainstream Version (MS-CISSAR) 
- Elementary (Carta, Greenwood, Schulte, et al., 1987). Due to their detailed nature, the 
taxonomies for each instrument is provided in the Appendix C. 

Because of their ecobehavioral taxonomies, these instruments provide information on child behav- 
ior, caregiver/teacher behavior, and the immediate situational context using a momentary time sam- 
pling observation procedure (see Greenwood, Carta et al., 1990). As a result, the information pro- 
duced taps a number of natural “active ingredients” with respect to planning intervention goals and 
procedures. We briefly mention some of these ingredients. 



Infant/Toddkrs 

Childrens’ social and communicative behaviors, their participation in object play, manipulation of 
objects, and engagement in typical home, and child care activities are an array of developmentally 
significant behaviors an interventionist may wish to increase. Similarly, the level of infant distress, social 
rejection, and aggression in early childhood are behaviors that an interventionist may well want to 
decrease. 

Caregiver behavior such as comforting touch and face-to-face positioning (Crittendon & Bonvillian, 
1984; Field, 1994) are considered mediators of social and nonsocial engagement (Colombo & 
Horowitz, 1987). Caregivers vocalizing behaviors (imitation and narrating- Comfort & Farran, 1994) 
and amount of parents talk to their child is related to increased infant vocalization and later language 
development (Hart & Risley, 1995; Walker, Greenwood, Hart, & Carta, 1994). Other caregiving 
behaviors such as close involvement, verbal responsiveness, and affectionate behavior describe experi- 
ences related to developmental resilience in at risk children (Bradley et al., 1994). 
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Preschoolers . Attention to and engagement in learning tasks including fine motor, preacademic, and 
talk are considered important to the development of early cognitive and social functioning (Green- 
wood, Hart, Walker, &C Risley, 1994). It is known that preschool teachers who follow the lead of the child 
in order to expand on the talk and reinforce child talk accelerate language development (Hart, 1985). 

The environmental arrangement of day care and classrooms settings affect more cooperative play in 
enclosed spaces rather than in large open spaces (Carta, Sainato, &C Greenwood, 1988). Materials and 
classroom seating arrangements are known to affect a range of early skills including independent work 
and transition (McEvoy, 1990). Play patterns have also been found to be associated with the presence 
or absence of certain types of materials (Carta, Sainato, & Greenwood, 1988). For example, crayons, 
art materials, books and puzzles have been found to promote solitary or parallel play, whereas, dolls 
and pretend play areas are associated with cooperative play. 



Early Elementary 

Engagement in active academic responding during classroom instruction is a well-known ingredient in 
accelerating learning and academic achievement in school-aged children (e.g., Greenwood, 1996a; 
Greenwood, Carta, & Atwater, 1991, Hoge, 1985;). Teachers who use group as well as individual 
questioning strategies, increase opportunities to respond and students’ reading, writing, and talk about 
subject matter. Teachers who use materials and tasks that support the use of literacy skills promote 
reading and engagement in academic responding. 



Order (f interpretation 



Ecobehavioral assessment is used to tap these and other active ingredients. Interpretation of these data 
is best addressed in terms of a layered set of questions beginning with child behavior, caregiver, and 
situational context (Greenwood, Peterson, & Sideridis, 1995). This order reflects the centrality of child 
behavior in the context of the caregiver and the situation, and the importance of each in formulating a 
solution. For example: 

1. What is the profile of child behavior overall; and during specific conditions of activities? 

2. What is the profile of caregiver behavior overall and during specific conditions of activities? 

3. Is the caregiver interacting with the child at all? 

4. How is the caregiver interacting with the child? 

5. Where is the caregiver located in relationship to the child? 

6. How is the immediate setting organized in terms of architecture, activities, and persons? 

7. Are desired activities and materials present in the setting and in use? 

8. Do the activities and caregivers behavior suggest a particular treatment or instructional 
strategy? 



9. 



Is a specific instructional strategy previously taught to the caregiver in evidence and to what 
degree compared to previous observations? 
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Research plans and strategies for shaping this instrument from pre-existing, 

RELATED TOOLS INTO FINAL PRODUCTS 

Research plans and strategies for the Ecobehavioral Assessment (EA) component involve four major 
steps. Software development, validation, treatment validation, and related studies (see Original Pro- 
posal, pp. 69-95). 

fostnmentSoftw^ 

Activities in Year 1 have focused on creating a software system for conducting EA. We are building a 
user-friendly software interface for these ecobehavioral instruments to make them accessible and usable 
by practitioners. Based on the Ecobehavioral Assessment Software Systems (EBASS Version 3.0 for 
DOS) (Greenwood, Carta, Kamps, & Delquadri, 1994), the new software, EBASS98, is currently in 
development and will be a MS Windows program developed in C++. Instruments supported in the 
EBASS98 will be: CIRCLE (Level 1 and 2) for infants and toddlers and their environments, ESCAPE 
for preschoolers and the preschool environment, and MS-CISSAR for primary grades and their school 
environments. Compared to EBASS Version 3.0, the CIRCLE instruments are a new addition and 
extend the range of assessment to infants and toddlers. These instruments are the product of previous 
development and all have been used in major research studies involving children, caregiving and teach- 
ing, and intervention. A table of technical information supporting each is provided in Appendix C. 

EBASS98 will support use by practitioners using portable computers as data collection devices. In 
addition to assisting the timing and collection of observation data, it will support: learning an observa- 
tion taxonomy and data collection through a tutorial system, calibration and interobserver agreement, 
data analysis and interpretation, and data management. 

Vdidatkm 

In the validation studies planned, we seek to increase each instruments generality across children (i.e., 
differing ages and types of disability). Using EBASS98, we will conduct a study of the reliability, 
validity, and cost of its application across ages and types of children by instrument (Greenwood, Carta, 
Kamps, & Arreaga-Mayer, 1990). 

The primary research questions to be addressed are: 

1. What is the reliability (percentage agreement and Kappa) across raters/scorers/judges within 
instruments (CIRCLE, ESCAPE, and MS-CISSAR) at each age group and type of student? 

2. What is its concurrent validity with measures of growth and development and does it differentiate 
relevant comparison groups (e.g., types of students at the same age, program variations such as 
NAEYC approved vs. uncertified programs, or interventions provided by licensed vs. unlicensed 
teachers)? 

3. What is the social validity of this approach? 

4. What is the cost of this software system, including training, data collection, data management/ 
analysis? 
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We will employ methods of percentage agreement and Kappa in order to examine observer agree- 
ment and reliability (question 1). We will employ Pearson r and Multiple regression to examining 
concurrent relationships between observation variables and growth and development measures 
(question 2). We will employ simple descriptive statistics and graphic displays to depict parents, 
teachers, and practitioners level of importance assigned to the information contained in the observa- 
tions to be completed (question 3), and to describe costs in terms of time and dollars of this approach 
(question 4). 

Trmlmentvcilidity 

Studies completed in this section address the ability of eco behavioral data from the three instruments to 
contribute measurably to the acceleration of growth and development. Using EBASS98, we will con- 
duct a two-year longitudinal study of the benefits of this information and its contribution to childrens 
growth and development. The primary research question will be: 

1 . Does this type of assessment lead to improved outcomes for children and programs? 

2. What is the social validity and cost of using the procedures, as described previously. 

Participants in this study will be drawn from each age cohort and type. Pairs of subjects matched on 
type of disability will be assigned at random to groups using versus not using observational assessment 
to guide changes in their programs. We expect 3 pairs (6) per type (8) and age group (4) or approxi- 
mately 192 children in all who will be followed for two years. The general longitudinal design will 
involve comparisons of the growth and development of children whose caregivers and teachers use the 
observational assessment to inform and guide changes in their program versus those who do not. The 
parents and teachers of those children in the observation this group will be trained and will use obser- 
vational data provided by the research team to guide decisions of when and how to change the inter- 
vention program. Both groups will be assessed using the methods of growth and development con- 
structed by other investigators in the Institute. 

The primary research question will be addressed using simple descriptive statistics and graphs of 
trends combined with Hierarchical Linear Modeling (HLM) techniques (Bryk &C Raudenbush, 1992). 
The primary analysis will test the null hypothesis that there is no difference in initial levels (intercept) 
and growth (slopes) for children in each age cohort (Infant = 48, Toddler = 48, Preschool =48, Pri- 
mary Grade = 48). Rejection of the null hypothesis, finding a significant difference in slopes for the 
ESA Group is expected; however, and such a finding will support the benefit of this approach. Analysis 
of growth for individual pairs of students will dramatically illustrate the generality of this effect across 
types of children and disabilities at different ages. 

Related investigations . Additional studies will be conducted in collaboration with the entire team of 
investigators. It is anticipated that opportunities will arise in which EBASS98 will be used by col- 
leagues in their studies to further address the issue of the benefits of this type of ESA. EBASS98 will 
be prepared from the outset for dissemination to these colleagues, and their access and use supported. 
We expect the opportunity of using ecobehavioral assessment in conjunction with other ESAs (i.e., 
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program features, and/or activity/curriculum-based methods) will occur in ways that will lead to new 
studies and new strategies of developing effective interventions. 

Examples and Illustrations 

What is the focus of intervention (i.e., what behaviors are to be taught/learned)? The behaviors to be 
taught and promoted in a child can often be targeted for intervention using ecobehavioral assessment 
(EA). EA supports development of short-term goals related to changes in the behavior of children and 
their caregivers? Related to any IGDI, is a class of related engaged skills/behaviors that support or 
detract from its acceleration. An example is the impact of daily passage reading activities and reading 
practice (as in ClassWide Tutoring, for example) on correct oral reading rate (an IGDI). These reading 
behaviors during instruction are necessarily distinguished from error patterns that typically reflect the 
basis academic skills needed to be taught in reading, writing, or math that are best assessed with cur- 
riculum-based assessment. 

For example, the general class of behaviors considered to be “attention” or “academic responding” 
can be targeted for improvement and linked to an instructional intervention. Similarly, the class of 
behaviors considered to compete with attention or academic responding (e.g., acting-out, looking 
around) can be targeted for reduction in support of a specific IGDI (e.g., accelerating segmentation 
rate, oral reading rate, math digit rate). Additionally, engagement in highly specific behaviors such as 
academic talk, verbalizations, babbles, etc., can be targeted for improvement as they relate to accelera- 
tion in other IGDIs (e.g., rate of vocabulary growth, social interaction). Time engaged in specific fine 
or large motor tasks (an ESA) can be a targeted for improvement by an interventionist, as a means of 
improving: the rate of pulling, pushing, or placing components; growth in lifting strength, or duration 
under conditions of exertion, or other such motor IGDIs. 

What caregiving/ treatment/ instructional strategies and arrangements will be used for the intervention ? 
Selecting, designing, and implementing an intervention strategy can be supported by EA in a number 
of important ways. One way (and perhaps the least complex) is to change an element of the ecology to 
better support change in student behavior and opportunity to respond. For example, because EA 
reveals that a student is provided reading instruction for less time than planned, provision is made to 
increase this time with certainty. Or, because EA reveals that a student lacks the requisite materials, 
provision is made to provide them. Or, because EA reveals that a child is out of view of the teacher, 
the child’s seating is relocated to improve ability to monitor his behavior. In each of these cases, 
because the interventions are made (as confirmed by quality of intervention measures - QOI), the child’s 
engaged behavior is improved (as confirmed by EA), and growth in the relevant IGDI is accelerated. 

A second way is to provide a strategy known to promote a specific behavior or class of behaviors 
because increasing them is known to promote growth in the IGDI of interest. For example, faced with 
lack of growth in correct oral reading rate (an IGDI), use of EA reveals a student with limited oppor- 
tunity to read text passages during typical reading instruction. Consequently, a decision is made to 
implement reading ClassWide Peer Tutoring (R-CWPT) or reading Peer-Assisted Learning Strategies 
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(PALS) both programs known to increase time spent practicing reading and reading fluency (e.g., 
Greenwood, 1996a). Because QOI confirms that CWPT was implemented and that the students daily 
time engagement in reading (oral and silent) has tripled over prior assessments, growth in the relevant 
IGDI evident. 

A third way is to use EA is to understand the function of problem behavior and intervene so as to 
promote appropriate behavior. Excessive problem behavior can reduce growth in an IGDI, and EA can 
be used to develop, implement, and monitor an intervention based on this understanding. For ex- 
ample, a child with a high rate of disruption and acting-out during reading may demonstrate much 
lower disruptive rates during prevocational instruction based on EA assessments. This finding leads to 
planning that makes reading instruction for this child much more like prevocational instruction (in 
terms of activities, materials, demands, difficulty, teacher behavior). QOI confirms that implementa- 
tion of the strategy has been implemented as planned, that student disruption has declined (EA), and 
that growth in the IGDI has resumed. 

Quality of Implementation 

What do we know about the active ingredients of this ESA that effects caregiving, teaching and learning? It 
is widely recognized that in order to reap the benefits of specific intervention practices, particularly 
those supported by research, they must be implemented with some degree of specificity and adherence 
to procedural steps, those that activate the “active ingredients” (Carta & Greenwood, 1989; Wolery & 
Holcombe, 1993). It is also known that variation from the intervention protocol produces variation in 
the effects produced, unfortunately in many cases, reducing the acceleration possible under full or 
complete implementation. Thus, quality of implementation assessment (QOI) provides a window on 
success implementing the intervention plan. It also plays an important role in training and sustaining 
the use of the intervention. QOI assessment is a basis for feedback and planning improvements in 
implementation quality. 

Research plans and strategies for shaping this instrument from pre-existing, related tools into final prod- 
ucts. Unlike the prior measures to be developed by the Institute Team, QOI assessments are developed 
by the interventionist to reflect a specific intervention and its ingredients. Users will develop their own 
instruments (e.g., checklists, based on procedures under development). Based on prior successful models 
(Abbott et ah, 1998; Greenwood, Carta, et ah, 1991; Wolery & Holcombe, 1993), objective methods 
for constructing these instruments will be developed using a combination of content analysis and 
sequence mapping to the relevant intervention protocol/procedures to be implemented and assessed. 

Examples and Illustrations 

In a study of the effects of consultation on kindergarten teachers’ use of story book reading strategy and 
its effects on students’ learning of basic concepts print, Walton (1998) reported that after training and 
coaching feedback, teachers increased from an average of 2 of these instructional behaviors prior to train- 
ing to 9 or more afterwards, and they maintained use over a school year with only monthly feedback. As 
can be seen, the checklist was developed to directly represent the content and sequence of the procedure. 
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Shared Book Experience Teacher Implementation Checklist ( Walton , 1998) 

Before Reading 

□ 1. Teacher activates prior knowledge. 

□ 2. Teacher points to picture on the cover and asks a question. 

□ 3. Teacher points to the title of the book as she reads it. 

□ 4. Teacher asks students to predict what the story might be about. 

□ 3. Before beginning to read, the teacher makes sure everyone can see the book. 
During Reading 

□ 6. Teacher reads with enthusiasm. 

□ 7. Teacher points to each word as she reads. 

□ 8. Teacher points to pictures. 

□ 9. Teacher asks questions about — 

□ pictures □ predictions 

□ characters □ setting 

□ 10. For repetitive or predictable stories, teacher accepts all students attempts to read. 



Connecting Print and Language 

□ 11. Teacher connects print and language by doing two or more of the following — 

□ masks or frames a word 

□ asks a student to find a word that begins with a specific letter 

□ asks students to identify a letter or letter sound 

□ asks students to find word in text or match a word on an index card with word in text 

□ uses words or letters from story for writing activity 

□ emphasizes concepts of print — 

□ lowercase letters □ upper case letters □ spacing □ left-to-right progression 

□ first word on a page □ return sweep □ period □ question mark 

Total II 1 
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Discussion 



Discussed in this report, is a set of intervention planning tools (program features, activity/curricu- 
lum-based, and ecobehavioral assessments) for use in a comprehensive system of individual growth and 
development, birth to age 8. These tools for exploring and evaluating intervention solutions for 
individual children are part of larger comprehensive system involving measures of individual growth 
and development (IDGI’s) and a decision-making framework designed to monitor progress and 
support the planning of care and teaching. 

Compared to measures of child progress, measures of potential solutions focus on a child’s strengths 
and weaknesses in the curriculum to identify skills mastered and skills to be taught, child/caregiver 
behaviors most likely to enable the learning of these skills, and elements of the child’s learning envi- 
ronment most responsible for teaching these skills and supporting their generalization. 

Described was the conceptual basis and unique role of these tools in the selection, planning, 
and implementation of interventions designed to accelerate individual progress. The plans and strate- 
gies for moving each of these forms of assessment and specific instruments from preexisting to final 
products was described and illustrations of their use provided. 
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Appendix A 



Program Features 



Classroom Observation 

1. Are there an abundance of books and reading materials that are rotated frequently? Are there a 
variety of books? Are there writing utensils? 

2. Is there a high quality reading corner with soft and carpeted areas? Is there a cozy, comfortable, 
and quiet place where children can go to read (e.g., a partitioned off, attractive, and well-stocked 
library corner with carpet, pillows, etc.)? Is it compatible with adjoining areas (e.g., not directly 
next to a block building center)? 

3. Is there an integration of literature and literacy materials during free-play activities? Are there 
toys and literacy materials visually and physically accessible to children during free choice 
activities? Are play centers enriched with familiar literacy materials (e.g., pencils, markers, paper, 
recipes, grocery lists, phone books, menus, mail, books, magazines, labeled product containers, 
etc.)? Are there toys and materials which promote language and cooperative play (e.g., balls, dress- 
up clothes, games, construction materials)? 

4. Is there an adult model who is frequently writing, reading and using print in the classroom? 

Are teachers incorporating opportunities for language and social skills within the routines you 
observe (e.g. discussion during storytime, having children use words to express wants, 
frustrations, etc.)? Do teachers model language and literacy use (Interactions, writing, reading, 
etc.)? Do teachers use expansion and asking questions (why, what, who, where) techniques to 
promote language? Are there both teacher and child-directed activities? 

5. Are there displays of childrens work, labeling of classroom items and schedule? Are there 
written displays around the room: childrens work, charts, lists, directions, captioned artwork, 
items labeled (e.g. desk, chair, sink), childrens names, etc.? Are these displays at childrens eye- 
level? Is a classroom schedule visible to all children? Does it use both pictures and words? Is there 
a specified time for a language activity (e.g. storytime, letter naming, etc.)? Are language goals 
posted? 

6. What is the class size and ratio of teacher to children during the observation? 



Immediately after observing: During your observation, did a speech/language professional provide 
services to any child or children? If so, was the service provided within the classroom? Did the speech/ 
language professional work primarily with the children or with the staff? Did you observe opportuni- 
ties for children to interact with children with various levels of language ability? 
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Teacher Survey 

Classroom Information 

1. Number of students in your classroom. 

2. Range of students’ ages 

3. Number of trained teachers 

Years of education 

Number of aides 

Other adults 

4. Are there students in your classroom with IEPs who are receiving speech 

and language services? If yes, how many? 

What types of services? 

Where do these services take place? 

5. How many other students are receiving any type of special services? 

What types of services? 

Where do the various services take place? 
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1 . Are materials rotated; for example old books removed and new ones added? 

If yes, approximately how often? 

2. Do you have a designated “reading” or book area? 

If yes, how many children are allowed in the area at one time? 

3. Are there times that materials in areas are limited to encourage children to share or request more 

materials? If so, is there a clear method provided for doing so? 

4. Does the schedule allow for planning and preparation time before the children arrive? 

5. Does the schedule allow a regular time for staff to review each days events and childrens progress 

and needs? 

6. Does the schedule allow a regular time for staff to evaluate and plan ongoing programs together? 

7. Does the schedule allow sufficient time for informal contact with parents, as needed, 

(not including scheduled teacher/parent conferences) 

8. Are the days/weeks activities planned ahead of time yet are flexible enough to allow for the 

emergence of new themes and directions for exploration? 

9. Do daily plans and individual activity plans include back up plans? 

10. Is there a variety of activities planned each day? 

1 1 . Do the children participate in creating the classroom rules? 

12. Are children taught the rules? 

13. How do you use materials to encourage language? 



14. 



Are there opportunities for children to interact in small groups with children of varying language 
ability? 
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Activity-Based Assessment 
Scripts 



Script for Block Activity 


Goals 


Descri ption/Sequence 


Prompts (Verbal + Physical) 


IICOGD2. Places 
objects in series 
according to length or 
size 


If child is aligning objects, 
present 3 objects that can 
be placed in a series 
according to length or size 


P & V: Model putting objects in series, 
“Look, 1 put these in order. Can you put 
yours in order? 


ICOGE2. Uses object 
to obtain another object 


Set the container with 
material set B above child’s 
reach. “Please get that box 
on the counter for me.” 


V: “How can you reach those blocks? 


IFMB2. Assembles 
toy/objects that require 
putting pieces-together 


“Thank you, now you can 
play with what is inside.” 


V: Can you put the person in the car? 
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Ecobehavioral Assessment 

Contents of This Appendix 
Instrument Taxonomies 

1) CIRCLE 1 -Infant 

2) CIRCLE 2-Toddler 

3) ESCAPE- Preschool (3 to 5 years) 

4) MS-CISSAR-Elementary (Kg - grade 6) 

Summary of Instruments’ Technical Indicators 
EBASS98 Data Collection Screen for EBASS 
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Discipline Older Child 

Gentle Rocking 
Vistibular Movement 
Routine Handling 
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